The present invention relates generally to methods of data structure characterization, indexing, storage and retrieval; and more particularly comprises a method for enabling easy characterization, storage and retrieval of multi-dimensional data structures involving use of a translation, rotation and scaling invariant index which results from concatenating a series of Eigenvalue calculation mediated index elements determined at a plurality of hierarchical data depth levels. A variation thereof pre-selects pixels in data structures which have associated therewith significant intensity and/or color gradient(s) with respect to surrounding pixels, in, for instance, xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d direction(s), then utilizes said pre-selected pixels in the data structure characterizing spatial first and second central moment calculations. Data structure characterizing vector(s) are constructed from spatial data structure first central moment values, and non-degenerate parameter(s) determined from second central moment calculations, as well as from, when applicable, frequency-of-occurance integer counts of said pre-selected pixels.
Data structures, such as two dimensional pixel arrays, are being generated at an ever increasing rate. For instance, algorithm generated and scanned computer screen images, X-ray, CT, MRI and NASA satellite, space telescope and solar explorer systems generate thousands of images every day. To make optimum use of said images, however, convenient methods of data characterization, storage and retrieval are required. For example, a medical doctor might obtain an X-ray image of a patient""s chest but has to rely on xe2x80x9cdiagnostic artxe2x80x9d to arrive at a diagnosis. Were it possible to determine an index which characterizes said X-ray image and also enable easy storage and retrieval thereof, it would be possible to compare said index to a catalog of indices of various X-ray images which are known to be associated, with various healthy or pathologic conditions. Thus diagnosis could be moved toward the very desirable goal of being objectively definite in a mathematical sense.
Continuing, it must be understood that conventional data bases are stored as text with organization being in terms of fields and values. Examples are business product, customer lists, sales data etc. To retrieve such data a user must issue a query in text format, similar to what is done in natural languages. It is essentially impossible to use such an approach to store and retrieve the contents of most data images, for example, because there is no convenient manageable way to describe such data images in terms of said fields and values. Data Images are instead typically stored in the form of compressed digital files of hundreds of thousands of binary numbers, and said storage technique does not facilitate easy image characterization, storage and retrieval. And, while it is possible to describe a data image with a text Index, to examine the data image still requires that the data associated with said Index be retrieved. It is also possible to assign an arbitrary serial number to a data image to facilitate data storage and retrieval, but under this approach the serial number provides no insight to the image and again, to examine data image, requires accessing the image data per se.
A preferred approach to the characterization of data images, which provides an index for use in storage and retrieval thereof, is to base the index on features in the data image. To arrive at such an index, however, is typically computationally complex, requiring hundreds of thousands of calculations. That is, determination of said index must typically be extracted from a data image xe2x80x9coff-linexe2x80x9d. Characteristic indices so determined are called xe2x80x9cimage indicesxe2x80x9d, and ideally render a concise description, not only of an image color and intensity content on a row and column basis , but also of the nature and shape of objects therein. A problem arises, however, in that many image features can not be easily described. Geometric shapes in a data image, for example, can require a combination of text annotation and numeric values and often the result is not at all concise.
Continuing, it must be understood that conventional data are stored as text, with organization being in terms of alphebetic and numerical value fields. Examples are business product, customer lists, sales data etc. To retrieve such data a user must issue a query text format, similar to what is done in natural languages. It is, however, essentially impossible to use such an approach to store and retrieve the contents of most data structures, (two dimensional picture graphic data images for example), because there is no convenient manageable way to describe such data structures in terms of said alphabetic and numerical value fields. Data Structures are instead typically stored in the form of compressed digital files of hundreds of thousands of binary numbers, and said storage technique does not facilitate easy data structure indexing, characterization, storage and retrieval. And, while it is possible to describe a data structure with a text Index, to examine the data structure data still requires that the data associated with said index be retrieved. It is also possible to assign an arbitrary serial number to a data structure to facilitate data storage and retrieval, but under this approach the serial number provides no insight to the data, and again, to examine data structure, requires accessing the data per se.
A preferred approach to the characterization of data structures, which provides an index for use in storage and retrieval thereof, is to base the index on features in the data structure. To arrive at such an index, however, is typically computationally complex, requiring hundreds of thousands of calculations. That is, determination of said index must typically be extracted from a data structure xe2x80x9coff-linexe2x80x9d. Characteristic indices so determined are called xe2x80x9cstructure indicesxe2x80x9d, and ideally render a concise description, not only of a structure color and intensity content on a row and column basis, but also of the nature and shape of objects therein. A problem arises, however, in that many data structure features can not be easily described. Geometric shapes in a data structures, for example, can require a combination of text annotation and numeric values and often the result is not at all concise.
Relevant considerations in developing an approach to extracting xe2x80x9cimage indicesxe2x80x9d from a data image or data structure include:
1. Uniquenessxe2x80x94different images/data structures should have different associated indices, (ie. an image index should be non-degenerate);
2. Universalityxe2x80x94image/data structure indices must be extractable from essentially any kind of image to be characterized, stored and retrieved by use thereof;
3. Computationxe2x80x94image/data structure indices must be easily computed from any data structure to be characterized, stored and retrieved by use thereof, and computation complexity should be kept to a minimum possible;
4. Concisenessxe2x80x94image/data structure indices must concise and easy to store;
5. Invariancexe2x80x94descriptive features in an image/data structure must tolerate change of scale, rotation and translation transformations, image object position shifting, calibration of color and pixel intensity and return essentially unchanged image indices;
6. Noise resistantxe2x80x94random noise entry to image/data structure should not significantly change the image index extracted therefrom.
Previous attempts at extracting an index for an image/data structure have focused on use of:
pixel intensity and color distributions, (see an article titled xe2x80x9cQuery By Image And Video Content: The QBIC Systemxe2x80x9d), IEEE Trans. on Computers, (Sep. 1995));
pixel texture patterns (see a book titled xe2x80x9cDigital Image Processingxe2x80x9d, Gonzales, Addison-Wesley Pub. (1992)); and
edge and boundary-line shapes, (see a book titled xe2x80x9cDigital Image Processing And Computer Visionxe2x80x9d, Schalkoff, John Wiley and Sons, (1989)),
etc. as the basis of approach. These techniques are mainly based on the calculation of the statistics of a data image in a pixel arrangement. Said techniques often lack universality in that they work when applied to a certain type of data image, but not when applied to other types of data images. Moreover, many previous approaches are not image transformation invariant and do not tolerate entry of noise.
Continuing, one approach which provides a rotationally invariant result is termed xe2x80x9cEqual Angular Samplingxe2x80x9d. Said method provides a concatenation of numbers which are distances from a centroid in a data image to an intersection point with an object boundary. Said technique encounters problems, however, where objects with irregular shapes, with concave boundaries and/or wherein holes are encountered.
The use of Moment Invariants to describe the geometrical shape features of data images was proposed more than thirty (30) years ago by Hu in an article titled xe2x80x9cVisual Pattern Recognition By Moment Invariantsxe2x80x9d, IRE Trans. on Information Theory, IT-8, (Feb. 1963). The method is based in modeling an image as a physical object with masses distributed in two dimensional space. It typically treats the pixel intensities as the probability distribution value of the object masses. The central moments in various orders are calculated on distributions. A set of central moment invariants is derived from making algebraic combinations of the central moments. The most important property of he technique is that the resulting descriptive quantities are transformation invariant, (ie. the central moment invariants remain unchanged when the image undergoes scaling, rotation, translation, intensity, or color platter changes). See an article titled xe2x80x9cRecognitive Aspects Of central moment Invariantsxe2x80x9d, by Abu-Mostafa et al., IEEE Trans. on Pattent Analysis and Mach Intell., Vol. PAMI-6, No. 6, (Nov. 1984).
Additional references of interest are:
xe2x80x9cImage Analysis Via the General Theory Of Momentsxe2x80x9d, Teague, J. Opt. Soc. America, Vol. 70, No. 8, (Aug. 1980), which discloses that a 2D shape obtained from central moment invariants defined on the second central moments can be viewed as an elliptic approximation of the shape; and
xe2x80x9cA Transformation-Invariant Recursive Subdivision Method For Shape Analysisxe2x80x9d, Zhu and Poh, IEEE Proc. of the 9th Int. Conf. on Pattern Recog., Rome, Italy, (Nov. 14-17, 1988).
It is to be appreciated that Statistical and central moment-based descriptions of data can distinguish data images at only very rough levels. That is, an image index associated with a data image is not unique and could be arrived at by analysis of an alternative data image. In addition, the computations involved in practicing said Statistics and central moment-based approaches can be complicated and time consuming and can require both character and numeric symbols in a resultant image index. And the use of the central moment invariant approach can involve the computation of an image index in high orders.
With the present invention in mind a Search of Patents was performed with the result being that very little was found. A Patent to Windig, U.S. Pat. No. 5,841,891 is disclosed, however, as it identifes the calculation of Eigenvalues, but in a method for enhancing images. A Patent to Shimura et al., U.S. Pat. No. 5,644,756 is also identified as it describes generating calculated feature data for identifying images, with application in image identification. A Patent, U.S. Pat. No. 5,608,862 to Enokida is disclosed as it describes development of a tag which indicates the length of data in hierarchically coded image data. U.S. Pat. No. 5,572,726 to Hasuo is disclosed as it describes an index image for use in retrieval of data. A Patent to Tsujumura et al., U.S. Pat. No. 5,586,197 is disclosed as it describes using color as a basis of searching for a data image in an image database. Finally, a U.S. Pat. No. 4,742,558 to Ishibashi et al. is disclosed use of a hierarchical structure for use in image retrieval and display.
An updated Patent Search conducted with a present invention pixel pre-selection methodology in mind, has provided the following additional Patents:
U.S. Pat. No. 5,748,777 to Katayama which describes apparatus and methodology for extracting outline data;
U.S. Pat. No. 4,896,364 to Lohscheller which describes a method of detecting boundary structures in a video signal;
U.S. Pat. No. 5,805,728 to Munesada et al., which describes an edge line measuring method;
U.S. Pat. No. 5,930,391 to Kinjo which describes a method of extracting an outer edge in an image;
U.S. Pat. No. 5,902,987 to Coffman et al. which describes a method and apparatus for locating the edges of patterns in a stored image;
U.S. Pat. No. 5,889,882 to Senn et al. which describes a method of determining a skin line in a digital medical image;
U.S. Pat. No. 5,828,771 to Bloomberg which describes an image processing technique which automatically analyzes an image to determine if it is an original or a copy.
U.S. Pat. No. 5,790,269 to Masaki et al. which describes a method and apparatus for extracting edge or contour information; and
U.S. Pat. No. 4,849,914 to Medioni et al. which describes a method and apparatus for registering color separation using edge based registration techniques.
In addition, the following publications are also known and disclosed:
xe2x80x9cQuery By Image And Video Contentxe2x80x9d, Flickner, IEEE Computer, pp. 23-30, (Sep. 1995);
xe2x80x9cCompact Representations Of Videos Through Dominant And Multiple Motion Estimationsxe2x80x9d, Sawhney and Ayer, IEEE Transactions On Pattern Analysis And Machine Intelligencexe2x80x9d, pp. 814-830, Vol. 18, No. 8, (Aug. 1996);
xe2x80x9cRecovering 3D Motion Of Multiple Objects Using Adaptive Hough Transformxe2x80x9d, ITian and Shah, EEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 19, No. 10, pp. 1178-1183, (Oct. 1997);
xe2x80x9cA Survey Of Technologies For Parsing And Indexing Digital Videoxe2x80x9d, Ahanger and Little, Visual Communication and Visual Representation, Vol. 7, No. 1, pp. 28-43, (1996);
xe2x80x9cReview Of Image And Video Indexing Techniquesxe2x80x9d, Idris and Panchanathan, Visual Communication and Image Representation, Vol. 8, No. 2, pp. 146-166, (1977); and
xe2x80x9cAnalysis Of Video Image Sequences Using Point And Line Correspondencesxe2x80x9d, Wang, Karandikar and Aggarwal, Pattern Recognition, PR(24), pp. 1065-1084, (1991).
Even in view of the prior art there remains need for a convenient method of characterizing two-dimensional data images, and multi-dimensional data structures, so that they can be easily indexed, stored and retrieved, which method reduces computational requirements as much as possible. Said method should provide an index which demonstrates Uniqueness, Universality, Computational Ease, Conciseness, Invariance to data image change of scale, rotation and translation of data image object, position shifting, linear calibration of color and pixel intensity, and Resistance to random noise. Additionally, said methodology should provide for tracking image changes in sequential images, such as in videos. In answer to the identified need, the present invention provides a method of extracting data indicies from a data structure which results from application of a pixel spatial data structure location based first central moment determined mean, and non-degenerate parameter(s) determined from second central moment values, in xe2x80x9cXxe2x80x9d, xe2x80x9cYxe2x80x9d and where a three-dimensional data structure is investigated, xe2x80x9cZxe2x80x9d orthogonally related directions. Additional benefit is identified as deriving from a frequency-of-occurance determination based, significant pixel identifying, pre-selection procedure, which results in said data index then being further comprised of typically orthogonally specific frequency integer counts of pre-selected pixels.
The Present Invention has as its focus the producing of an Index (I) which is Non-Degenerately, (ie. Uniquely in that each Index describes xe2x80x9cOne Imagexe2x80x9d at some Hierarchical Level), descriptive of a Multiple, (eg. Two), Dimensional Data Image (MDDI), which Index (I) is Image Rotation, Translation, Scaling, and Intensity, Color etc. invariant. (It is noted that Intensity and Color of, for instance, a Digital Data Image are typically combined to form a single value at each Pixel (x, y) location which is a function f(x,y), by for instance, a formula that has as input variables xe2x80x9camountsxe2x80x9d of Red, Green and Blue content as well as Intensity for each Pixel).
The present invention procedure involves finding, at each of a number of Hierarchical depth levels, Eigenvalues, which Eigenvalues can each be thought of as the Major (xcex1) and Minor (xcex2) Axes of a xe2x80x9cCharacteristic Virtual Ellipsexe2x80x9d which represents said Hierarchical depth level. (The term xe2x80x9cVirtualxe2x80x9d indicates that the Ellipse is not usually actually drawn, but rather can be helpfully visualized as descriptive of the geometry of the Data).
At the First Hierarchical Level, the Image as a whole is considered and subjected to Mathematical Techniques, which lead to determination of xcex11+xcex21, which xcex11+xcex21 are the major and minor axes of the Characteristic Virtual Ellipse. The Non-degenerate Index Component, (which is a functional focus of the present invention as applied in the context of the Eigenvalues), for said First Level is then calculated from an equation which returns a Non-Degenerate result, such as:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
Next, the Characteristic Virtual Ellipse Minor Axis Eigenvalue is typically considered as a dividing line between two xe2x80x9cSidesxe2x80x9d of the Data Image as a whole, and the data on each side thereof is considered separately, at what is the Second Hierarchical Data Depth Level, of the Procedure. For each of said two (2) xe2x80x9cSidesxe2x80x9d additional Eigenvalues, which are major and minor axes of characteristic virtual ellipses, are calculated and for each of said two (2) xe2x80x9cSidesxe2x80x9d, and Non-degenerate Index Components for said Second Level are then calculated from Non-Degenerate returning Equations, such as:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
An updated Non-Degenerate Index is formed by concatenation of the Three Index Components so far found, (ie. (I1) from the First Level Data (L1) Image as a whole plus two (I2 and I3) from the Second level (L2)):
I=I1 I2 I3, (or any combination or permutation thereof).
Next, Data in each xe2x80x9cSidexe2x80x9d identified at the Second Level (L2) is divided, typically around the Minor Characteristic Virtual Ellipse Axis thereby providing a Third Level with Four (4) Separate Sets of Data. Each of said Separate Sets of Data is then subjected to similar Mathematical Treatment to the end that Four additional Non-Degenerate Index Components (I4, I5, I6 and I7), are derived, and concatenated onto or with the Three Non-Degenerate Index Components (I1, I2 and I3) already present to provide a string of Seven Non-Degenerate Index Components comprising (I1, I2, I3, I4, I5, I6 and I7) in some functional permuted order.
The above procedure can be carried forward to as many Hierarchical Depth Levels as desired to provide a Non-Degenerate Index of a desired length and level of accuracy, even down to where a single Pixel distinguishes the Image defined by a produced Index (I).
(Notexe2x80x94known techniques identify Pixel Color and Brightness content and perhaps some shape content, but do not do so in a manner which is Image Rotation, Translation, Scaling, and Intensity, Color etc. invariant). At this point, educated reflection should result in realization that Linear Rotation or Translation or Scaling of a Data Image will not change Non-degenerate Index (I) element values produced by practice of the Present Invention, as the underlying Eigenvalues will not change. Similarly, if the same linear change is made to Intensity at all Pixels in an Image, no change will occur in calculated Non-degenerate Index element values.
It is also to be understood that while the above specifies the minor axis xcex2 of the characteristic virtual ellipse as a data dividing locus at each hierarchical data depth level after the first, it is possible to use the major axis xcex1. In addition, it is possible to use both the major and minor axes as data dividing loci and, for instance, at the second data depth level add four additional Index Elements. The third data depth level would then add sixteen additional Index Elements and so on.
In light of the foregoing, a number of variations of the present invention method are recited directly.
A very general present invention method of providing an identifying data set index (I) for an xe2x80x9cNxe2x80x9d dimensional data set consisting of a multiplicity of pixels, comprises the steps of:
a. determining Eigenvalues for essentially the entire data set, said Eigenvalues being a major axis and (Nxe2x88x921) minor axes of a characteristic virtual data set mathematical object, then calculating a first non-degenerate data set index element using a formula which operates on said major axis and at least one of said (Nxe2x88x921) minor axes;
b. dividing said essentially entire xe2x80x9cNxe2x80x9d dimensional data set into at least first and second data set parts about at least one axis selected from the group consisting of: (said major axis and said (Nxe2x88x921) minor axes), and for at least one of said at least first and second data set parts independently determining xe2x80x9cNxe2x80x9d Eigenvalues therefore, said xe2x80x9cNxe2x80x9d Eigenvalues being a manor axis and (Nxe2x88x921) minor axes of a mathematical object for said at least one of said at least first and second data set parts, and then calculating at least one additional non-degenerate data set index element using formula(s) which operate on said major axis and at least one of said (Nxe2x88x921) minor axes in said at least one of said at least first and second data set parts, and return a non-degenerate result; and
c. concatenating at least two resulting non-degenerate data set index elements in any functional permuted order to provide said identifying data set index (I).
Continuing, another recitation of a present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, said method comprises the steps of:
a. determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis and a minor axis of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said major and minor axes Eigenvalues;
b. dividing said essentially entire two dimensional data image into at least first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said at least first and second data image parts independently determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a first of said separate characteristic virtual ellipses, and major axis and a minor axis for a second of said separate characteristic virtual ellipses, and optionally independently determining a major axis and a minor axis for at least some of any additional data image parts, and then calculating at least second (I2) and/or third (I3) non-degenerate data image index elements determined from two of said at least two data image parts, using formulas which return a non-degenerate result;
c. concatenating at least two of said first (II), second (I2) and/or third (3) non-degenerate data image index elements in any functional permuted order to provide said identifying data image index (I).
Continuing, a more focused present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises the steps of:
a. determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex21) and a minor axis (xcex11) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using the formula:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
b. dividing said essentially entire two dimensional data image into first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said first and second data image parts independently determining Eigenvalues therefore, said Eigenvalues being a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, and then calculating second (I2) and third (I3) non-degenerate data image index elements using the formulas:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
c. concatenating said first, second and third non-degenerate data image index elements to provide said identifying data image index (I) by a selection from the group consisting of:
I=I1 I2 I3;
I=I1 I3 I2;
I=I2 I1 I3;
I=I2 I3 I1;
I=I3 I1 I2; and
I=I3 I2 I1.
Another present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises the steps of:
a. determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using the formula:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
b. dividing said essentially entire two dimensional data image into first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said first and second data image parts independently determining Eigenvalues therefore, said Eigenvalues being a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, and then calculating second (I2) and third (I3) non-degenerate data image index elements using the formulas:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
c. dividing said each of said first and second image parts from step b., reach about an axis selected from the group consisting of: (said minor axis and said major axis thereof), to produce third, forth, fifth and sixth image parts and for at least one of said third, forth, fifth and sixth image parts independently determining Eigenvalues thereof, said determined Eigenvalues being selected from the group consisting of:
a major axis (xcex14) and a minor axis (xcexA24) for the third of said separate characteristic virtual ellipses, and major axis (xcex15) and a minor axis (xcex25) for the forth of said separate characteristic virtual ellipses, a major axis (xcex16) and a minor axis (xcex26) for the fifth of said separate characteristic virtual ellipses, and major axis (xcex17) and a minor axis (xcex27) for the sixth of said separate characteristic virtual ellipses, and then calculating forth (I4), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data image index elements using as appropriate the formulas:             I      4        =                                                      (                                                λ                  14                                -                                  λ                  24                                            )                        2                                              λ              14                        +                          λ              24                                      ⁢                  xe2x80x83                ⁢                  I          5                    =                                    (                                          λ                15                            -                              λ                25                                      )                    2                                      λ            15                    +                      λ            25                                          I      6        =                                                      (                                                λ                  16                                -                                  λ                  26                                            )                        2                                              λ              16                        +                          λ              26                                      ⁢                  xe2x80x83                ⁢                  I          7                    =                                    (                                          λ                17                            -                              λ                27                                      )                    2                                      λ            17                    +                      λ            27                              
d. concatenating said first (I1), and at least one produced non-degenerate data index element(s) selected from the group consisting of said: (second (I2), third (I3), forth (14), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data index elements), in any functional permuted order, to provide said identifying data image index (I).
Said method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels can, in steps c. and d. thereof, provide that all of said first (I1), second (I2), third (I3), forth (I4), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data index elements, are calculated and concatenated, in any functional permuted order, to provide said identifying data image index (I).
Said present invention method, as is true for all variations thereof presented herein, can, as a preliminary step, subject the data to a pre-selection criteria to identify pixels which meet specified criteria. As is better described in the Section of this Specification titled xe2x80x9cData Pixel Pre-Selectionxe2x80x9d, a prefered pixel pre-selection criteria involves identifying and retaining pixels which define image feature edges, while eliminating pixels from a data set which are not associated therewith.
Such a modified method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels can be recited as comprising the steps of:
a. applying a pixel pre-selection criteria to eliminate pixels not associated with image feature edges;
b. determining Eigenvalues for essentially the entire remaining data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using the formula:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
c. dividing said essentially entire two dimensional data image into first and second data image parts about an axis selected from the group consisting of:
said minor axis;
said major axis; and
an axis intermediate thereto;
xe2x80x83and for at least one of said first and second data image parts independently determining Eigenvalues therefore, said Eigenvalues being a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, and then calculating second (I2) and third (I3) non-degenerate data image index elements using the formulas:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
d. concatenating at least two of said determined first, second and third non-degenerate data image index elements in any functional permuted order to provide said identifying data image index (I).
Continuing, another variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises the steps of:
a. determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said Eigenvalues:
b. dividing said essentially entire two dimensional data image into first, second, third and forth data image parts using said minor axis and said major axis as dividing means, said first, second, third and forth data image parts being oriented in a first, second, third and forth quadrant pattern defined by said major and minor axes, in said two dimensional data image;
c. for at least one of said first, second, third and forth data image parts independently determining Eigenvalues of a characteristic virtual ellipse therefore, said Eigenvalues being selected from the group consisting of:
a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and a major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, a major axis (xcex14) and a minor axis (xcex24) for the third of said separate characteristic virtual ellipses, and a major axis (xcex15) and a minor axis (xcex25) for the forth of said separate characteristic virtual ellipses;
d. calculating at least one additional non-degenerate data image index element using formula(s) which operates on Eigenvalues corresponding to said at least one of said first, second, third and forth data image parts determined in step. c; and
e. concatenating at least two resulting non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
In said variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels wherein Eigenvalues for each of said first, second, third and forth data image parts are independently determined, said Eigenvalues being:
a major axis (xcex11) and a minor axis (xcex21) for the first of said separate characteristic virtual ellipses, and a major axis (xcex12) and a minor axis (xcex22) for the second of said separate characteristic virtual ellipses, a major axis (xcex13) and a minor axis (xcex23) for the third of said separate characteristic virtual ellipses, and a major axis (xcex14) and a minor axis (xcex24) for the forth of said separate characteristic virtual ellipses;
step d. involves calculating four additional non-degenerate data image index element using formula(s) which operate on Eigenvalues corresponding to said first, second, third and forth data image parts determined in step. c; and
e. concatenating said resulting five non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
Said variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels can further comprise dividing said each of said first, second, third and forth data image parts from step b., each about at least one axis selected from the group consisting of: (said minor axis and said major axis) thereof, to identify at least two, and possibly four, new data image parts in each of said first, second, third and forth data image parts; and for at least some of the new data image parts determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a characteristic virtual ellipse for said newly formed data image part(s); and concatenating at least some of the calculated non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
It is also to be appreciated that given a present invention index (I) as determined by any of the foregoing methods, it is possible to construct a data image or set by expanding each index element into a set of pixels, by an essential reversal of the recited procedural steps.
It is also noted that a the present invention comprises a method of identifying similarities in a first data set to those in a second data set. Said method comprises the steps of, for each said data set:
a. determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis and a minor axis of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said major and minor axes Eigenvalues; and
b. comparing said first (I1) non-degenerate data image index element for said first data set to that for said second data set.
Further, said method of identifying similarities in one a first data set to those in a second data set can comprise the steps of, for each said data set:
b. dividing said essentially entire two dimensional data image into at least first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said at least first and second data image parts independently determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a first of said separate characteristic virtual ellipses, and major axis and a minor axis for a second of said separate characteristic virtual ellipses, and optionally independently determining a major axis and a minor axis for at least some of any additional data image parts, and then calculating at least second (I2) and/or third (I3) non-degenerate data image index elements, determined from two of said at least two data image parts, using formulas which return a non-degenerate result; and
c. comparing at least said second (I2) non-degenerate data image index element for said first data set to that for said second data set.
Of course, the method of identifying similarities in one a first data set to those in a second data set can proceed to comparison of the third (I3) and additional non-degenerate data image index elements obtained from each of the data sets.
Continuing, with the foregoing basic methodology in mind, it is disclosed that a preferred version of the present invention methodology, while again begining with the providing of a data structure, and proceeding to determine at least one data structure characterizing vector(s) which facilitates indexing, storage and retrieval of the data structure, additionally includes data point pre-selection. Data structure characterizing vector(s) is/are then constructed from typically orthogonally specific xe2x80x9cfrequency-of-occurancexe2x80x9d integer counts of pixels which are selected upon meeting an associated intensity or color or combined intensity and color gradient criteria, and for each orthogonal direction, calculated pixel spatial data structure location based first central moment, and non-degnerate parameter(s) determined from second central moment values.
The reason pixels pre-selection is desirable is that processing each pixel in a data structure when calculating associated spatial first and second central moments takes time, and because where region(s) of essentially similar pixel intensity or color or combined intensity and color content exists in a data structure there is limited value in performing spatial central moment calculations at every present pixel, (unless, perhaps, the data structure, beneficially visualized as two-dimensional picture graphic, has very few high gradient xe2x80x9cedgesxe2x80x9d of depicted objects therein), the present invention teaches that pixel values in a data structure should be subjected to a xe2x80x9cfrequency of occurancexe2x80x9d pre-selection procedure prior to calculating any data structure pixel xe2x80x9cspatial locationxe2x80x9d based first central moment and non-degenerate parameter(s) determined from second central central moment values.
The presently disclosed version of the invention teaches a pre-selection of all pixels in a data structure, (eg. two-dimensional picture graphics), to determine which thereof are in regions in which change:                     ∇        x            =                        δ          ⁢                      xe2x80x83                    ⁢                      f            ⁡                          (                              x                ,                y                            )                                                δ          ⁢                      xe2x80x83                    ⁢          x                      ;          xe2x80x83        ⁢                  ∇        y            =                        δ          ⁢                      xe2x80x83                    ⁢                      f            ⁡                          (                              x                ,                y                            )                                                δ          ⁢                      xe2x80x83                    ⁢          y                      ;    or                    ∇        x            =                        1          /          3                ⁢                              ∑                          i              =                              -                1                                                    +              1                                ⁢                      xe2x80x83                    ⁢                      [                                          f                ⁡                                  (                                                            x                      -                      1                                        ,                                          y                      +                      i                                                        )                                            -                              f                ⁡                                  (                                                            x                      +                      1                                        ,                                          y                      +                      i                                                        )                                                      ]                                ;                      ∇        y            =                        1          /          3                ⁢                              ∑                          i              =                              -                1                                                    +              1                                ⁢                      xe2x80x83                    ⁢                      [                                          f                ⁡                                  (                                                            x                      +                      i                                        ,                                          y                      -                      1                                                        )                                            -                              f                ⁡                                  (                                                            x                      +                      i                                        ,                                          y                      +                      1                                                        )                                                      ]                                ;  
is occuring therein, as regards intensity or color or a combination of intensity and color. (Note that a parameter representing color can be calculated using a non-limiting formula such as:
(C)=(aRed+bGreen+cBlue),
where aRed, bGreen and cBlue are measures of amounts of Red, Green and Blue present at a pixel location. A combined intensity and color parameter can be arrived at by, for instance, adding (C) to an Intensity representing (I) value). Then performance of spatial distribution central moment calculations is undertaken to provide first and second central moment values, utilizing only the pre-selection procedure identified pixels.
The pre-selection procedure then involves checking for reasonably large gradient change in each of the xe2x80x9cXxe2x80x9d and the xe2x80x9cYxe2x80x9d, (and xe2x80x9cZxe2x80x9d directions in a 3-D data structure), at each pixel in a data structure, and then includes only pixels whereat a reasonably large gradient is determined to exist in an xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d direction, in spatial Pixel location distribution based first and second central moment calculations.
For emphasis, (as it is a focal point of the present invention methodology), it is again stated that the pixel pre-selection procedure determines what pixels have associated therewith sufficiently large gradients of intensity and/or color, while the calculation of first and second central moment based values, (and non-degenerate parameters based on the later), is based on spatial locations of the pixels pre-selected on said basis of intensity and/or colo gradients.
The present invention method further provides for defining ranges of gradients in each of the xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d directions at pixels, and determining the number of pre-selected pixels which are in each range. A data structure, (eg. two dimensional picture-graphics or multi-dimensional data structure), characterizing, identifying, vector is then constructed for each gradient range, which characterizing vector comprises, for instance, the Integer Number Count of pre-selected pixels in the xe2x80x9cXxe2x80x9d, and/or the Integer Number Count of pre-selected pixels in xe2x80x9cYxe2x80x9d, and/or the Integer Number Count of pre-selected pixels in xe2x80x9cZxe2x80x9d, directions which have gradients in the selected range, and which vector further comprises spatial first central moments for each of the xe2x80x9cXxe2x80x9d, xe2x80x9cYxe2x80x9d and xe2x80x9cZxe2x80x9d directions, as well as a non-degenerate parameter constructed from spatial second cental central moment dependent Eigenvalues:   s  =                    (                              λ            1                    -                      λ            2                          )            2                      λ        1            +              λ        2            
calculated utilizing only the pixels with an intensity or color or combination intensity and color gradient value in an identified range. An example vector, the meaning of which will become more clear later herein is:
xe2x80x9c#X1xe2x80x9d,
xe2x80x9c#Y1xe2x80x9d,
xe2x80x9c#Z1xe2x80x9d,
MEAN1 xe2x80x9cXxe2x80x9d,
MEAN1 xe2x80x9cYxe2x80x9d,
MEAN1 xe2x80x9cZxe2x80x9d,
PARM1.
Again, the described procedure can repeated for multiple such gradient ranges of intensity or color, or both, (say gradient ranges wherein the gradient is between arbitrary units of say (1 and 10), or between arbitrary units of (11 and 100), or between arbitrary units of (100 and 500)), and the various Integer count number of pixels which fall thereinto in xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d directions, and spatial location first central moment, and a non-degenerate parameter constructed from spatial location second central moment values calculated utilizing only the pixels with a gradient value in a range combined to provide a unique data structure (eg. picture-graphics), indexing vector. One such vector can be determined for each range of intensity or color, or both, gradients, and the resulting vectors concatenated to provide a more definite data structure identifying index. It is noted that typical practice provides that once 50% of the pre-selected pixels have been utilized in the various ranges, no additional groupings are formed or required to provide additional vector(s) which are concatonated into the data structure identifying index.
Where more than one grouping of pre-selected pixels is made based upon application of multiple grouping defining gradient selection xe2x80x9cfrequency-of-occurancexe2x80x9d bases then, for a two-dimensional data array, a corresponding more than one, at least one five (5) member, vector:
(xe2x80x9c#Xxe2x80x9d, xe2x80x9c#Yxe2x80x9d, MEAN xe2x80x9cXxe2x80x9d, MEAN xe2x80x9cYxe2x80x9d, PARM)
as described later herein., will result. Say, for example, three ranges of gradients are defined and applied during the pixel pre-selection procedure, then three, at least five element vectors will result. Where multiple, at least five element, vectors result, the present invention teaches that said multiple vectors are concatonated in some order to provide a data structure characterizing indexing parameter. As an example, if three gradient ranges are defined with arbitrary units of:
(1 to 10);
(11 and 100);
(100 and 500)),
then three (3) five (5) element vectors, (eg. V1, V2 and V3):
V1=(xe2x80x9c#X1xe2x80x9d, xe2x80x9c#Y1xe2x80x9d, MEAN1 xe2x80x9cXxe2x80x9d, MEAN1 xe2x80x9cYxe2x80x9d, PARM1); and
V2=(xe2x80x9c#X2xe2x80x9d, xe2x80x9c#Y2xe2x80x9d, MEAN2 xe2x80x9cXxe2x80x9d, MEAN2 xe2x80x9cYxe2x80x9d, PARM2); and
V3=(xe2x80x9c#X3xe2x80x9d, xe2x80x9c#Y3xe2x80x9d, MEAN3 xe2x80x9cXXxe2x80x9d, MEAN3 xe2x80x9cYxe2x80x9d, PARM3);
can formed, (one per gradient range), and said vectors can be concatonated as a selection from the group consisting of:
V1 V2 V3;
V1 V3 V2;
V2 V1 V3;
V2 V3 V1;
V3 V1 V2; and
V3 V2 V1.
to form an a data structure characterizing identifier.
Additionally, the present invention teaches that where at least three (3) different intensity, or color, or combination intensity and color based gradient ranges are defined, and where a data structure characterizing indexing vector is calculated for each, then information in each said of three of said at least three data structure characterizing indexing vectors can be applied to provide three distinct xe2x80x9cpointsxe2x80x9d, which three distinct xe2x80x9cpointsxe2x80x9d can be considered as being the verticies of a triangle. Said triangle defining xe2x80x9cpointsxe2x80x9d can be re-calculated at sequential frames in a video, for instance, and sequential changes in said triangle defining xe2x80x9cpointsxe2x80x9d identified with Rotation, Translation and Scaling of objects in the data structure being analyzed. A method of tracking changes in a multi-dimensional data structure can be recited as comprising the steps of:
a. providing a first frame of a sequence of frames in a multi-dimensional data structure, each of said frames being comprised of pixels and with each of said pixels there being associated intensity and/or color(s);
b. defining at least three selected ranges of gradient(s) and defining pixel pre-selection technique gradient range selection criteria, then based on a selection from the group consisting of:
intensity gradiant;
color gradient; and
gradiant based on intensity and color;
determining pixels in said multi-dimensional data structure which demonstrate an intensity and/or color gradient within at least one of said at least three selected ranges of gradient(s); and
c. performing calculations on said pre-selection technique determined pixels to determine first and second central moment values and non-degenerate parameter(s) based upon said second central moment values, said first and second central moment values being based on spatial location distributions of said pixels identified by said pre-selection technique; separate calculations being performed utilizing pixels in each of said at least three ranges of gradients;
d. utilizing information from the separate calculations performed in three of said at least three ranges of gradients in step c. to form three points which are assigned as verticies of a triangle for said first frame of a sequence of frames in a multi-dimensional data structure comprising pixels;
e. providing a second frame of a sequence of frames in a multi-dimensional data structure, each of said frames being comprised of pixels and with each of said pixels there being associated intensity and/or color(s);
f. defining gradient range selection criteria substantially the same as in step b. and utilizing an intensity and/or color gradient determining pre-selection technique which is substantially the same as in step b. to determine pixels in said multi-dimensional data structure which demonstrate an intensity and/or color gradient within at least three of said at least three selected range of gradient(s), said at least three selected range of gradient(s) being substantially the same as in step b.; and
g. performing calculations on said step f. pre-selection technique determined pixels to determine first and second central moment values and non-degenerate parameter(s) based upon said second central moment values, said first and second central moment values being based on spatial location distributions of said pixels identified by said pre-selection technique; separate calculations being performed utilizing pixels in each of said at least three ranges of gradients;
h. utilizing information from the separate calculations performed in said three ranges of gradients in step g. to form three points which are assigned as verticies of a triangle for said second frame of a sequence of frames in a multi-dimensional data structure comprising pixels; and
i. comparing the results of practice of steps d. and h. to determine translation, rotation and scaling of objects identified in said first and second sequential frames of said multi-dimensional data structure.
Continuing, the present invention, in a very basic sense, is a method of characterizing data structures, (eg. two dimensional picture graphics), comprising:
1. utilizing an intensity and/or color gradient determining xe2x80x9cfrequency-of-occurancexe2x80x9d pre-selection technique to determine pixels in said data structure which should be utilized, (ie. are associated with xe2x80x9cedgesxe2x80x9d of forms in the data structure); and
2. performing calculations to determine first and second central moment values and non-degenerate parameter(s) based upon said second central moment values, said first central central moment values and second central moment values being based on spatial location distributions of said pixels identified by said xe2x80x9cfrequency of occurancexe2x80x9d pre-selection technique.
To the Inventor""s knowledge a two step procedure, combining xe2x80x9cfrequency-of-occurancexe2x80x9d pixel pre-selection followed by calculations to determine xe2x80x9cspatialxe2x80x9d first and second central moment values of said pre-selected pixels, has not been previously reported.
Application of present invention methodology to a two dimensional picture graphic, for instance, typically results in at least one five (5) member data structure characterizing index vector being produced, in which the members of the vector are:
a. number of pre-selected pixels in xe2x80x9cXxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
b. number of pre-selected pixels in xe2x80x9cYxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
c. a spatially based xe2x80x9cXxe2x80x9d direction calculated first central moment for the pre-selected pixels in the xe2x80x9cXxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
d. a spatially based xe2x80x9cYxe2x80x9d direction calculated first central moment for the pre-selected pixels in the xe2x80x9cYxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
e. a non-degenerate parameter calculated from spatially based second central moment.
As alluded to earlier herein, such a data structure characterizing indexing vector can be represented asxe2x80x9d
(xe2x80x9c#Xxe2x80x9d, xe2x80x9c#Yxe2x80x9d, MEAN xe2x80x9cXxe2x80x9d, MEAN xe2x80x9cYxe2x80x9d, PARM);
where:
xe2x80x9c#Xxe2x80x9d is the Integer number of pixels selected by applying the pre-selection frequency-of-occurance criteria to intensity or color or combination intensity and color gradients in the xe2x80x9cXxe2x80x9d direction;
xe2x80x9c#Yxe2x80x9d is the Integer number of pixels selected by applying the pre-selection frequency-of-occurance criteria to intensity or color or combination intensity and color gradients in the xe2x80x9cYxe2x80x9d direction;
Mean xe2x80x9cXxe2x80x9d is the spatial location based first central moment of the xe2x80x9c#Xxe2x80x9d pixels selected by applying the pre-selection frequency-of-occurance criteria to intensity or color or combination intensity and color gradients in the xe2x80x9cXxe2x80x9d direction;
Mean xe2x80x9cYxe2x80x9d is the spatial location based first central moment of the xe2x80x9c#Yxe2x80x9d pixels selected by applying the pre-selection frequency-of-occurance criteria to intensity or color or combination intensity and color gradients in the xe2x80x9cYxe2x80x9d direction; and
PARM is a value calculated from spatial location based second central moment dependent Eigenvalues, utilizing pixels selected by applying the pre-selection frequency-of-occurance criteria to intensity or color or combination intensity and color gradients in the xe2x80x9cXxe2x80x9d and xe2x80x9cYxe2x80x9d directions, such as provided by the formula:   s  =                    (                              λ            1                    -                      λ            2                          )            2                      λ        1            +              λ        2            
xe2x80x83where (xcex1) and (xcex2) are said Eigenvalues.
Continuing, a present invention method of characterizing data structures can then be recited as comprising the steps of:
a. providing an at least two dimensional data structure comprising pixels with each which are associated intensity and/or colors;
b. defining a gradient range selection criteria;
c. utilizing a pixel pre-selection technique based on a selection from the group consisting of:
an intensity gradiant;
a color gradient; and
a gradiant based on intensity and color;
xe2x80x83to determine pixels in said data structure which demonstrate an intensity and/or color gradient which meet said gradient range selection criteria; and
d. performing calculations on said pre-selection technique determined pixels to determine selection(s) from the group consisting of:
first central moment(s);
second central moment(s); and
non-degenerate parameter(s) based upon said second central moment(s)
said first and second central moments being based on spatial location distributions of said pixels identified by said pre-selection technique.
Said present invention method can be further identified as operating on a data structure which is a two dimensional picture graphic, and in which second central moments and non-degenerate parameter(s) based upon said second central moments are determined, and in which at least one five (5) member data structure characterizing index vector is produced, the members of said at least one five (5) member data structure characterizing index vector being comprised of, in any order:
a. number of pre-selected pixels in xe2x80x9cXxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
b. number of pre-selected pixels in xe2x80x9cYxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
c. spatially based xe2x80x9cXxe2x80x9d direction calculated first central moment for the pre-selected pixels in the xe2x80x9cXxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
d. spatially based xe2x80x9cYxe2x80x9d direction calculated first central moment for the pre-selected pixels in the xe2x80x9cYxe2x80x9d direction, said selection criteria being based on a gradient xe2x80x9cfrequency-of-occurancexe2x80x9d basis;
e. a non-degenerate parameter calculated from spatially based second central moment dependent Eigenvalues.
Further, said data structure characterizing indexing vector can be represented as
(xe2x80x9c#Xxe2x80x9d, xe2x80x9c#Yxe2x80x9d, MEAN xe2x80x9cXxe2x80x9d, MEAN xe2x80x9cYxe2x80x9d, PARM);
where the vector entries were described earlier herein.
A present invention method of characterizing data structures can also be recited as comprising:
a. providing a multi-dimensional data structure comprising pixels with each of which are associated intensity and/or colors;
b. defining gradient range selection criteria;
c. utilizing a pixel pre-selection technique based on a selection from the group consisting of:
an intensity gradiant;
a color gradient; and
a gradiant based on intensity and color;
xe2x80x83to determine pixels in said multi-dimensional data structure which demonstrate intensity and/or color gradient(s) which meet said gradient range selection criteria; and
d. performing calculations on said pre-selection technique determined pixels to determine first and second central moment and non-degenerate parameter(s) based upon said second central moment values, said first and second central moment values being based on spatial location distributions of said pixels identified by said pre-selection technique, separate calculations being performed utilizing pixels in each of said at least one range(s) of gradients.
Continuing, calculation of second central moment related Eigenvalues was described earlier herein, where the invention which has as its focus the producing of an Index (I) which is Non-Degenerately, (ie. Uniquely in that each Index describes xe2x80x9cOne Imagexe2x80x9d at some Hierarchical Level), descriptive of a Multiple, (eg. Two), Dimensional Data Image (MDDI), which Index (I) is Image Rotation, Translation, Scaling, and intensity, color etc. invariant. (It is noted that intensity and color of, for instance, a Digital Data Image are typically combined to form a single value at each Pixel (x, y) location which is a function f(x,y), by for instance, a formula that has as input variables xe2x80x9camountsxe2x80x9d of Red, Green and Blue content as well as intensity for each Pixel).
The invention as earlier disclosed is a procedure which involves finding, at each of a number of Hierarchical depth levels, Eigenvalues, which Eigenvalues can each be thought of as the Major (xcex11) and Minor (xcex21) Axes of a xe2x80x9cCharacteristic Virtual Ellipsexe2x80x9d which represents said. Hierarchical depth level. (The term xe2x80x9cVirtualxe2x80x9d indicates that the Ellipse is not usually actually drawn, but rather can be helpfully visualized as descriptive of the geometry of the Data).
At the first Hierarchical Level, the Image as a whole is considered and subjected to Mathematical Techniques, which lead to determination of (xcex11) which (xcex21) are the major and minor axes of the Characteristic Virtual Ellipse. The Non-degenerate Index Component, (which is a functional focus of the present invention as applied in the context of the Eigenvalues), for said first Level is then calculated from an equation which returns a Non-Degenerate result, such as:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
While the presently disclosed version of the invention methodology dous not typically so provide, the version presented earlier next typically provides that the Characteristic Virtual Ellipse Minor Axis Eigenvalue is considered as a dividing line between two xe2x80x9cSidesxe2x80x9d of the Data Image as a whole, and the data on each side thereof is considered separately, at what is the second Hierarchical Data Depth Level, of the Procedure. For each of said two (2) xe2x80x9cSidesxe2x80x9d additional Eigenvalues, which are major and minor axes of characteristic virtual ellipses, are calculated and for each of said two (2) xe2x80x9cSidesxe2x80x9d, and Non-degenerate Index Components for said second Level are then calculated from Non-Degenerate returning Equations, such as:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
An updated Non-Degenerate Index is formed by concatenation of the Three Index Components so far found, (ie. (I1) from the first Level Data (L1) Image as a whole plus two (I2 and I3) from the second level (L2)):
I=I1 I2 I3, (or any combination or permutation thereof).
Next, Data in each xe2x80x9cSidexe2x80x9d identified at the second Level (L2) is divided, typically around the Minor Characteristic Virtual Ellipse Axis thereby providing a Third Level with Four (4) Separate Sets of Data. Each of said Separate Sets of Data is then subjected to similar Mathematical Treatment to the end that Four additional Non-Degenerate Index Components (I4, IS, I6 and I7), are derived, and concatenated onto or with the Three Non-Degenerate Index Components (I1, I2 and I3) already present to provide a string of Seven Non-Degenerate Index Components comprising (I1, I2, I3, I4, I5, I6 and I7) in some functional permuted order.
The above procedure can be carried forward to as many Hierarchical Depth Levels as desired to provide a Non-Degenerate Index of a desired length and level of accuracy, even down to where a single Pixel distinguishes the Image defined by a produced Index (I).
(Notexe2x80x94known techniques identify Pixel color and Brightness content and perhaps some shape content, but do not do so in a manner which is Image Rotation, Translation, Scaling, and intensity, color etc. invariant). At this point, educated reflection should result in realization that Linear Rotation or Translation or Scaling of a Data Image will not change Non-degenerate Index (I) element values produced by practice of the Present Invention, as the underlying Eigenvalues will not change. Similarly, if the same linear change is made to intensity at all Pixels in an Image, no change will occur in calculated Non-degenerate Index element values.
It is also to be understood that while the above specifies the minor axis xcex21 of the characteristic virtual ellipse as a data dividing locus at each hierarchical data depth level after the first, it is possible to use the major axis xcex11. In addition, it is possible to use both the major and minor axes as data dividing loci and, for instance, at the second data depth level add four additional Index Elements. The third data depth level would then add sixteen additional Index Elements and so on.
As should now be appreciated, the present invention is, in a very basic sense, a method of characterizing data structures, (eg. picture graphics), comprising, in any functional permuted order:
a. providing a data structure, (eq. a picture graphic);
b. defining gradient range selection criteria and utilizing a pixel pre-selection technique based on a selection from the group consisting of:
an intensity gradiant;
a color gradient; and
a gradiant based on intensity and color;
xe2x80x83to determine pixels in said multi-dimensional data structure which demonstrate intensity and/or color gradient(s) which meet said gradient range selection criteria and which are to be utilized; and
c. defining ranges of gradients;
d. for each gradient range, performing calculations to determine first central moment values in at least two typically orthogonal directions, and second central moment values and non-degenerate parameter(s) based upon said second central moment values, said first and second central moment values being based on spatial location distributions of said pixels identified by said xe2x80x9cfrequency of occurancexe2x80x9d pre-selection technique; and
e. forming a data structure characterizing index vector for at least one of said defined gradient ranges.
A preferred data structure characterizing indexing vector comprisesxe2x80x3:
(xe2x80x9c#Xxe2x80x9d, xe2x80x9c#Yxe2x80x9d, MEAN xe2x80x9cXxe2x80x9d, MEAN xe2x80x9cYxe2x80x9d, PARM);
the explanations of the entries in said data structure characterizing indexing vector having been described earlier herein. And, where more than one data structure characterizing indexing vector is calculated, (each data structure characterizing indexing vector corresponding to a range of intensity or color or combination intensity and color gradient), said multiple data structure characterizing indexing vectors can be concatenated to provide a more definite data structure characterizing index.
In light of the foregoing, a number of variations of the present invention method are recited directly. It is to be appreciated that each of the following recitations includes the pixel pre-selection methodology just described. That is, rather than act on all pixels in a data structure, pixels in a data structure are pre-selected prior to practicing the methodology, to provide a beginning reduced pixel containing data structure.
A very general method of providing an identifying data structure index (I) for an xe2x80x9cNxe2x80x9d dimensional data structure consisting of a multiplicity of pixels comprises a pre-selection of pixels in said xe2x80x9cNxe2x80x9d dimensional data set based upon the meeting of an intensity and/or color gradient determining pre-selection criteria to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing said pre-selected pixels, determining Eigenvalues for essentially the entire data structure, said Eigenvalues being a major axis and (Nxe2x88x921) minor axes of a characteristic virtual data structure mathematical object, then calculating a first non-degenerate data structure index element using a formula which operates on said major axis and at least one of said (Nxe2x88x921) minor axes;
b. dividing said essentially entire xe2x80x9cNxe2x80x9d dimensional data structure into at least first and second data structure parts about at least one axis selected from the group consisting of: (said major axis and said (Nxe2x88x921) minor axes), and for at least one of said at least first and second data structure parts, utilizing said pre-selected pixels, independently determining xe2x80x9cNxe2x80x9d Eigenvalues therefore, said xe2x80x9cNxe2x80x9d Eigenvalues being a major axis and (Nxe2x88x921) minor axes of a mathematical object for said at least one of said at least first and second data structure parts, and then calculating at least one additional non-degenerate data structure index element using formula(s) which operate on said major axis and at least one of said (Nxe2x88x921) minor axes in said at least one of said at least first and second data structure parts, and return a non-degenerate result; and
c. concatenating at least two resulting non-degenerate data structure index elements in any functional permuted order to provide said identifying data structure index (I).
Continuing, another recitation of a present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, said method comprises a pre-selection of pixels in said two dimensional data set based upon the meeting of an intensity and/or color gradient determining pre-selection criteria to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing said pre-selected pixels, determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis and a minor axis of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said major and minor axes Eigenvalues;
b. dividing said essentially entire two dimensional data image into at least first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said at least first and second data image parts, utilizing said pre-selected pixels, independently determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a first of said separate characteristic virtual ellipses, and major axis and a minor axis for a second of said separate characteristic virtual ellipses, and optionally independently determining a major axis and a minor axis for at least some of any additional data image parts, and then calculating at least second (I2) and/or third (I3) non-degenerate data image index elements determined from two of said at least two data image parts, using formulas which return a non-degenerate result;
c. concatenating at least two of said first (I1), second (I2) and/or third (3) non-degenerate data image index elements in any functional permuted order to provide said identifying data image index (I).
Continuing, a more focused present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises a pre-selection of pixels in said two dimensional data set based upon the meeting of an intensity and/or color gradient determining pre-selection criteria to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing said pre-selected pixels, determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using the formula:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
b. dividing said essentially entire two dimensional data image into first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said first and second data image parts, utilizing said pre-selected pixels, independently determining Eigenvalues therefore, said Eigenvalues being a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, and then calculating second (I2) and third (I3) non-degenerate data image index elements using the formulas:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
c. concatenating said first, second and third non-degenerate data image index elements to provide said identifying data image index (I) by a selection from the group consisting of:
I=I1 I2 I3;
I=I1 I3 I2;
I=I2 I1 I3;
I=I2 I3 I1;
I=I3 I1 I2; and
I=I3 I2 I1.
Another present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises a pre-selection of pixels in said two dimensional data set based upon the meeting of an intensity and/or color gradient determining pre-selection criteria to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing said pre-selected pixels, determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using the formula:       I    1    =                    (                              λ            11                    -                      λ            21                          )            2                      λ        11            +              λ        21            
b. dividing said essentially entire two dimensional data image into first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said first and second data image parts, utilizing said pre-selected pixels, independently determining Eigenvalues therefore, said Eigenvalues being a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, and then calculating second (I2) and third (I3) non-degenerate data image index elements using the formulas:       I    2    =                                          (                                          λ                12                            -                              λ                22                                      )                    2                                      λ            12                    +                      λ            22                              ⁢              xe2x80x83            ⁢              I        3              =                            (                                    λ              13                        -                          λ              23                                )                2                              λ          13                +                  λ          23                    
c. dividing said each of said first and second image parts from step b., each about an axis selected from the group consisting of: (said minor axis and said major axis thereof), to produce third, forth, fifth and sixth image parts and for at least one of said third, forth, fifth and sixth image parts, utilizing said pre-selected pixels, independently determining. Eigenvalues thereof, said determined Eigenvalues being selected from the group consisting of:
a major axis (xcex14) and a minor axis (xcex24) for the third of said separate characteristic virtual ellipses, and major axis (xcex15) and a minor axis (xcex25) for the forth of said separate characteristic virtual ellipses, a major axis (xcex16) and a minor axis (xcex26) for the fifth of said separate characteristic virtual ellipses, and major axis (xcex17) and a minor axis (xcex27) for the sixth of said separate characteristic virtual ellipses, and then calculating forth (I4), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data image index elements using as appropriate the formulas:             I      4        =                                                      (                                                λ                  14                                -                                  λ                  24                                            )                        2                                              λ              14                        +                          λ              24                                      ⁢                  xe2x80x83                ⁢                  I          5                    =                                    (                                          λ                15                            -                              λ                25                                      )                    2                                      λ            15                    +                      λ            25                                          I      6        =                                                      (                                                λ                  16                                -                                  λ                  26                                            )                        2                                              λ              16                        +                          λ              26                                      ⁢                  xe2x80x83                ⁢                  I          7                    =                                    (                                          λ                17                            -                              λ                27                                      )                    2                                      λ            17                    +                      λ            27                              
d. concatenating said first (I1), and at least one produced non-degenerate data index element(s) selected from the group consisting of said: (second (I2), third (I3), forth (I4), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data index elements), in any functional permuted order, to provide said identifying data image index (I).
Said method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels can, in steps c. and d. thereof, provide that all of said first (I1), second (I2), third (I3), forth (I4), fifth (I5), sixth (I6) and seventh (I7) non-degenerate data index elements, are calculated and concatenated, in any functional permuted order, to provide said identifying data image index (I).
Another variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels, comprises a pre-selection of pixels in said two dimensional data set based upon the meeting of an intensity and/or color gradient determining pre-selection criteria to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing said pre-selected pixels, determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis (xcex11) and a minor axis (xcex21) of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said Eigenvalues:
b. dividing said essentially entire two dimensional data image into first, second, third and forth data image parts using said minor axis and said major axis as dividing means, said first, second, third and forth data image parts being oriented in a first, second, third and forth quadrant pattern defined by said major and minor axes, in said two dimensional data image;
c. for at least one of said first, second, third and forth data image parts, utilizing said pre-selected pixels, independently determining Eigenvalues of a characteristic virtual ellipse therefore, said Eigenvalues being selected from the group consisting of:
a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and a major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, a major axis (xcex14) and a minor axis (xcex24) for the third of said separate characteristic virtual ellipses, and a major axis (xcex15) and a minor axis (xcex25) for the forth of said separate characteristic virtual ellipses;
d. calculating at least one additional non-degenerate data image index element using formula(s) which operates on Eigenvalues corresponding to said at least one of said first, second, third and forth data image parts determined in step. c; and
e. concatenating at least two resulting non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
In said variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels wherein Eigenvalues for each of said first, second, third and forth data image parts are independently determined, said Eigenvalues being:
a major axis (xcex12) and a minor axis (xcex22) for the first of said separate characteristic virtual ellipses, and a major axis (xcex13) and a minor axis (xcex23) for the second of said separate characteristic virtual ellipses, a major axis (xcex14) and a minor axis (xcex24) for the third of said separate characteristic virtual ellipses, and a major axis (xcex15) and a minor axis (xcex25) for the forth of said separate characteristic virtual ellipses;
step d. involves calculating four additional non-degenerate data image index element using formula(s) which operate on Eigenvalues corresponding to said first, second, third and forth data image parts determined in step. c; and
e. concatenating said resulting five non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
Said variation on the present invention method of providing an identifying data image index (I) for a two dimensional data image consisting of a multiplicity of pixels can further comprise dividing said each of said first, second, third and forth data image parts from step b., each about at least one axis selected from the group consisting of: (said minor axis and said major axis) thereof, to identify at least two, and possibly four, new data image parts in each of said first, second, third and forth data image parts; and for at least some of the new data image parts, utilizing pre-determined pixels, determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a characteristic virtual ellipse for said newly formed data image part(s); and concatenating at least some of the calculated non-degenerate data index elements, in any functional permuted order, to provide said identifying data image index (I).
It is also to be appreciated that given a present invention index (I) as determined by any of the foregoing methods, it is possible to construct a data image or structure by expanding each index element into a structure of pixels, by an essential reversal of the recited procedural steps.
It is also noted that a the present invention comprises a method of identifying similarities in a first data structure to those in a second data structure. Said method comprises a pre-selection of pixels in both data sets based upon the meeting of an intensity and/or color gradient determining pre-selection to determine pixels in said data structure which demonstrate an intensity and/or color gradient within a selected range of gradients;
said method further comprising the steps of:
a. utilizing pre-determined pixels, determining Eigenvalues for essentially the entire data image, said Eigenvalues being a major axis and a minor axis of a characteristic virtual ellipse, then calculating a first (I1) non-degenerate data image index element using a formula which operates on said major and minor axes Eigenvalues; and
b. comparing said first (I1) non-degenerate data image index element for said first data structure to that for said second data structure.
Further, said method of identifying similarities in a first data structure to those in a second data structure can comprise the steps of, for each said data structure:
b. dividing said essentially entire two dimensional data image into at least first and second data image parts about an axis selected from the group consisting of: (said minor axis and said major axis), and for each of said at least first and second data image parts, utilizing said pre-selected pixels, independently determining Eigenvalues therefore, said Eigenvalues being a major axis and a minor axis for a first of said separate characteristic virtual ellipses, and major axis and a minor axis for a second of said separate characteristic virtual ellipses, and optionally independently determining a major axis and a minor axis for at least some of any additional data image parts, and then calculating at least second (I2) and/or third (I3) non-degenerate data image index elements determined from two of said at least two data image parts, using formulas which return a non-degenerate result; and
c. comparing at least said second (I2) non-degenerate data image index element for said first data structure to that for said second data structure.
Of course, the method of identifying similarities in one a first data structure to those in a second data structure can proceed to comparison of the third (I3) and additional non-degenerate data image index elements obtained from each of the data structures.
While the foregoing has consistantly identified use of major and/or minor axis as data set division criteria, it is to be understood that while preferred, this is not required. That is, for instance, after determination of a major and minor axis for an entire data set, one can divide the data set around some axis which is intermediate to the major and minor axes. In such a case present methodology of providing an identifying data set index (I) for an xe2x80x9cNxe2x80x9d dimensional data set consisting of a multiplicity of pixels, comprises the steps of:
a. determining Eigenvalues for essentially the entire data set, said Eigenvalues being a major axis and (Nxe2x88x921) minor axes of a characteristic virtual data set mathematical object, then calculating a first non-degenerate data set index element using a formula which operates on said major axis and at least one of said (Nxe2x88x921) minor axes;
b. dividing said essentially entire xe2x80x9cNxe2x80x9d dimensional data set into at least first and second data set parts about at least one axis selected from the group consisting of: (said major axis; and said (Nxe2x88x921) minor axes; and an axis intermediate thereto), and for at least one of said at least first and second data set parts independently determining xe2x80x9cNxe2x80x9d Eigenvalues therefore, said xe2x80x9cNxe2x80x9d Eigenvalues being a major axis and (Nxe2x88x921) minor axes of a mathematical object for said at least one of said at least first and second data set parts, and then calculating at least one additional non-degenerate data set index element using formula(s) which operate on said major axis and at least one of said (Nxe2x88x921) minor axes in said at least one of said at least first and second data set parts, and return a non-degenerate result; and
c. concatenating at least two resulting non-degenerate data set index elements in any functional permuted order to provide said identifying data set index (I).
Where the data set is two dimensional and the mathematical object is an ellipse, the Eigenvalues will be the major and minor axes. In that case, a data set can be divided around either the major and minor axis, or some axis formed intermediate thereto.
It is further to be understood that while the preferred present invention mathematical object utilized to model data is an ellipse which presents with major and minor axes Eigenvalues, any functional permuted mathematical object from which can be extracted Eigenvalues can be utilized in practice of the method of the present invention, in any of its variations.
The foregoing recitation of variations on the method of the present invention which are within the scope thereof are, of course, demonstrative, and not limiting. Other variations of present invention methodology are possible and can be arrived at by combining elements of various recited step sequences in other combinations and/or orders etc.
The present invention will be better understood by reference to the Detailed Description of this Disclosure in conjunction with the accompanying Drawings.
It is therefore the primary purpose of the present invention to provide a method which enables easy characterization, storage and retrieval of multi-dimensional data structures, comprising use of a translation, rotation and scaling invariant index which results from concatenating a series of Eigenvalue calculation mediated index elements, determined at a plurality of hierarchical data depth levels.
It is another primary purpose of the present invention to provide a method which enables easy comparison of at least first and second multi-dimensional data structures, comprising use of translation, rotation and scaling invariant indicies which result from, for each of said at least first and second multi-dimensional data structures, concatenating a series of Eigenvalue calculation mediated index elements, determined at a plurality of hierarchical data depth levels.
It is another primary object and/or purpose of the present invention to teach a method which pre-selects pixels in data structures to determine those pixels which demonstrate significant data structure intensity and/or color gradients in xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d direction(s), (tyically the result of depicted-object-edge-effects in the data structure), and utilizes said pre-selected pixels in data structure characterizing spatial first and second central moment calculations, to the end that data structure characterizing vector(s), are constructed from typically orthogonally specific frequency integer counts of said pre-selected pixels, as well as spatial data structure first monent mean(s), and non-degenerate parameter(s) determined from second central moment dependent Eigenvalues calculated utilizing pre-selected pixels with data structure intensity and/or color gradients in xe2x80x9cXxe2x80x9d and/or xe2x80x9cYxe2x80x9d and/or xe2x80x9cZxe2x80x9d direction(s) within one or more designated range(s).
It is yet another object and/or purpose of the present invention to provide a method which enables reconstruction of data sets from calculated Indicies which characterize multi-dimensional data structures, comprising use of a translation, rotation and scaling invariant index which results from concatenating a series of Eigenvalue calculation mediated index elements, determined at a plurality of hierarchical data depth levels.
Other objects and/or purposes of the present invention will become obvious by referral to the Specification text and claims.